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CLAIMS 

What is claimed is: 

1 . A computer-based system comprising: 

a computer system executing task computing, thus enabling users to define tasks by 
combining available functionality and to execute such tasks. 

2. The computer-based system of claim 1 , wherein available functionality originates 
in devices, computing applications and electronic services available through remote procedure 
calls including Web Services, UPnP, CORBA, RMI, RPC, DCE, DCOM or comprises previously 
defined tasks. 

3. The computer-based system of claim 2, wherein all available functionality is 
abstracted to the user as a service and each service is expressed in a service description 
language. 

4. The computer-based system of claim 3 wherein 
each service has at least one semantic description associated with it, 

each semantic description can be provided by any combination of the creator of the service, the 
owner of the service or some other third party 

and the service can be made available or unavailable by making available or unavailable one or 
more of its semantic descriptions. 

5. The computer-based system of claim 4 further comprising: 
means for discovering available services; 

means for filtering the services based upon relevance of the discovered services to the 
user and her context; 

means for specifying the task possibly in collaboration with the user; 
means for executing the specified task; and 

optionally means for creating, removing, and managing services. 

6. The computer-based system of claim 5 wherein the specified tasks are saved, 
automatically, or as instructed by the user, as new available services. 
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7. The computer-based system of claim 6, wherein the previously specified tasks 
saved as services are available to the user who created them, to all other users, or to any user- 
defined, or pre-defined group of users. 

8. The computer-based system of claim 7, wherein the means for 
discovering services utilize a service discovery protocol, including any one or any combination 
of the following: 

UPnP, UDDI, Local Service Repository, Jini, Bluetooth SDP, Rendezvous, and InfraRed 

(IR). 

9. The computer-based system of claim 8, wherein the means for filtering 
discovered services include any one or any combination of the following: 

User profile, Task at hand, User device characteristics, User location, User motion status, 
User network connectivity, User specified keywords, features of a set of services when 
considered as a whole, individual service features. 

1 0. The computer-based system of claim 9, wherein the means for specifying a task 
include any one or any combination of the following: 

a planning-based, automated system, or 

an interactive user interface that supports any one or any combination of the following 
elements: visual, voice, text, Braille, tactile 

1 1 . The computer-based system of claim 1 0, wherein the means for executing a 
specified task utilizes a service execution mechanism including any one or any combination of 
the following: Web Services calls, UPnP actions, CORBA, RMI, RPC, DCE, DCOM, InfraRed 
(IR), Local function call, and Local object call. 

1 2. The computer-based system of claim 1 1 , wherein the means for creating, 
removing, and managing services include any one or any combination of the following: 
Graphical user interface, service execution, device interaction, events from the operating 
system of the user-operated device and applications executing on the user-operated device. 

1 3. The computer-based system of claim 1 2, further comprising: 
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a module for discovering services; 
a module for filtering services; 

a module for automatically specifying tasks according to user's goals or for assisting the 
user in specifying tasks by providing the user with information about how the services 
comprising the tasks can be combined; 

a module for assisting the user with executing tasks, by invoking the services that the 
task is composed of and providing the user with means for interacting with the invoked services 
as they are executed, and 

a User Interface that assists the user with Service Discovery, Service Filtering, Service 
Composition, Service Execution and the saving of defined tasks for future use 

14. The computer-based system of claim 13, wherein the modules execute on a single 
computing device or wherein the modules, or subcomponents of the modules, are distributed 
across multiple computing devices and are accessible by programmatic interfaces that are 
accessible to the User Interface that is preferably running on the device operated by the user. 

15. The computer-based system of claim 12, wherein the user engages in Service 
Discovery, followed by Service Filtering, followed by Task Specification, followed by Task 
Execution. 

16. The computer-based system of claim 15, wherein the user while engaged in Task 
Specification iterates one or more times through Service Discovery, or Service Filtering, or both. 

17. The computer-based system of claim 16, wherein the user while engaged in Task 
Execution iterates one or more times through Service Discovery, or Service Filtering, or Task 
Specification, or all three, or any combination of any successive two. 

1 8. The computer-based system of any one of claims 15, 16, and 1 7, further 
comprising a User Interface that supports the workflows of Service Discovery, Service Filtering, 
Service Specification and Service Execution. 
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1 9. The computer-based system of claim 1 8, wherein the User Interface is a 
graphical one and comprises any combination of: 

a Discovery Pane for displaying discovered services and filtered services; 
a Details Pane for displaying additional information of a selected service from the 
Discovery Pane; 

a Composition Pane for displaying listings of matched services from Discovery Pane; 

a Construction Pane for constructing a composition of services; 
an Information Pane for displaying general information about the processes of the tool, including 
information about the progress of the execution; 

a Save Pane for saying a composition of services; 
wherein all or a subset of the panes are simultaneously displayable to the user and actions on 
each of the panes results in automatically updating the related information displayed in other 
panes. 

20. The computer-based system of claim 19, wherein the user can move to the 
Construction pane by either selecting a service from Discovery pane and choosing the construct 
function, or by selecting a pair of matched services and choosing the construct function. 

21 . The computer-based system of claim 19, wherein the user can add or remove 
services from the currently composed task in the Construct pane by choosing the plus or minus 
function for each possible position that an insertion or deletion is possible, as is automatically 
determined by the back-end system. 

22. The computer-based system of claim 19, wherein the user can execute a 
specified task by choosing the execution function when that task becomes executable, as 
automatically determined by the underlying back-end system, while either at the Construct pane 
or the Composition pane. 

23. The computer-based system of claim 19, wherein the system automatically 
appends any, pre-defined or user-defined compatible composition of available services before 
or after a partially specified composition by the user and executes the complete composition 
whenever the user chooses the execution function. 
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24. The computer-based system of claim 19, wherein a composition to be saved can 
be extended so that its constituent services satisfy requirements relating to their properties. 

25. The computer-based system of claim 1 8, wherein the User Interface is a 
graphical one and comprises panes including any combination of: 

a Discovery Pane for displaying discovered services and filtered services; 
a Details Pane for displaying additional information of a selected service from the 
Discovery Pane; 

a Composition Pane for Displaying listings of matched services from Discovery Pane; 

a Construction Pane for constructing a composition of services; 

an Information Pane for displaying general information about the processes of the tool, 
including information about the progress of the execution; 

a Save Pane for saving a composition of services in terms of criteria; 
wherein at any given time only one of the tabs is visible to the user and actions on each of the 
panes results in automatically updating the related information displayed in other panes. 

26. The computer-based system of claim 25, wherein the user can specify a task 
pane by either selecting a service from Discovery pane and choosing the construct function, or 
by selecting a pair of matched services from the Composition tab and choosing the construct 
function 

27. The computer-based system of claim 25, wherein the user can add or remove 
services from the currently composed task in the Construct pane by choosing the plus or minus 
function for each possible position that an insertion or deletion is possible, as is automatically 
determined by the back-end system. 

28. The computer-based system of claim 25, wherein the user can execute a 
specified task by choosing the execute function when that task becomes executable, as 
automatically determined by the underlying back-end system, while either at the Construct pane 
or the Composition pane. 
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29. The computer-based system of claim 25, wherein the system automatically 
appends any, pre-defined or user-defined compatible composition of available services before 
or after a partially specified composition by the user and executes the complete composition 
whenever the user chooses the execution function. 

30. The computer-based system of claim 25, wherein a composition to be saved can 
be extended so that its constituent services satisfy requirements relating to their properties. 

31 . The computer-based system of claim 18, wherein the User Interface is a 
graphical one and comprises a first pane for displaying discovered services and filtered services 
and a second pane for displaying the current composition as composed by the user and controls 
for going back and forth in the history of the composition and a control for executing composition 
whenever possible. 

32. The computer-based system of claim 31 , wherein the User Interface 
automatically places a selected service to the proper place of the composition sequence. 

33. The computer-based system of claim 32, wherein the User Interface will 
automatically complete the composition whenever possible, including the case when the 
available filtered services can only be combined in a single manner for a composition that can 
be executed. 

34. The computer-based system of either claim 18, wherein the User Interface is a 
web client. 

35. The computer-based system of claim 34, wherein as the task is executed the 
user's browser might pop-up a new window or re-direct the browser to a new link that allows the 
user to enter information related to the execution of the task or feedback regarding the executed 
task. 
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36. The computer-based system of claim 18, wherein the User Interface is a 
command line Interface, wherein each time the user types text that can be matched against 
discovered and/or filtered services, upon the user pressing a pre-assigned key, it inserts the 
matched service or, if multiple matches exist it provides a listing of them for the user to select by 
means of further typing text that matches one of the many services. 

37. The computer-based system of claim 36, wherein the sequence of user selected 
service comprise a sequence that corresponds to a task and can subsequently be executed by 
means of pressing a pre-assigned key. 

38. The computer-based system of claim 14, wherein the Discovery module, Filtering 
Module, Composition Module and Execution module and User Interface execute on the same 
computing device. 

39. The computer-based system of claim 38, wherein the Composition Module 
comprises an Inference Engine that can identify all possible pair wise combinations of services 
that can be successfully executed in a composition given semantic descriptions of the 
constituent services. 

40. The computer-based system of claim 38, wherein the Composition Module 
comprises an Inference Engine that can identify all possible compositions of services that can 
be successfully executed in their entirety given the semantic descriptions of the constituent 
services. 

41 . The computer-based system according to any one of claims 39 and 40, wherein 
the number of possible pair-wise matches or compositions of services that are returned by the 
Inference engine might be restricted by criteria that include an upper bound on computational 
resources, an upper bound on computing time, the number of services in a composition, or by a 
pre-determined total number of pair-wise matches or compositions to be computed 

42. The computer-based system of claim 41 , wherein the Inference Engine includes 
a composition logic wherein the composition logic is a rule, or a set of rules, or a set of logic 
statements. 
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43. The computer-based system of claim 38, wherein the discovery module uses 
local semantic service description (SSD) databases or accesses remote SSD databases and 
retrieves the SSDs of the found service from those databases, using the service's ID specific to 
the discovery mechanism through which the service is found. 

44. The computer-based system of claim 43, wherein the discovery mechanisms is 
UPnP and the ID is Unique Device Name (UDN) of UPnR 

45. The computer-based system of claim 38, wherein mechanism known as UPnP 
action is used to retrieve one or more pointers to the semantic description of the service. 

46. The computer-based system of claim 14, wherein the User Interface is running 
on a computing device that is different than the computing devices Discovery Module, Filtering 
Module, Composition Module and Execution module execute. 

47. The computer-based system of claim 46, wherein the User Interface is a web 
client (browser) that communicates with the Discovery, Filtering, Composition and Execution 
Modules using http. 

48. The computer-based system of claim 46, wherein the User Interface is an 
application that executes on the end-user's computing device and communicates with each of 
the Discovery, Filtering, Composition and Execution modules via the programmatic interface of 
these modules. 

49. The computer-based system of claim 14, wherein the end-user is using any of a 
desktop, laptop, pen computer, PDA or mobile phone. 

50. The computer-based system of claim 12, wherein a service may have one or 
more interfaces for the user to interact with the service prior and during its execution.. 

51 . The computer-based system of claim 14, wherein a service comprises any one 

of: 

"View on Display" which lets the user to view a user-specified document on a display 
and optionally control how it is shown; 
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"View on Multiple Displays" which lets the user to view a user-specified document on 

more than one display and optionally control how it is shown; 

"Map Viewer" which lets the user to view a map of a user-specified location and 

optionally control how it is shown; 

"Map Router" which lets the user to view a map or textual description of the route from/to 

a user-specified location from/to a predefined or a user-specified location optionally control how 

it is shown; 

"Instance Providing Service" which provides, when executed, a semantic instance where 
a semantic instance is an instance of a class described in an ontology; 

"Print" which lets the user to print a user-specified document or an semantic instance in 
a specific format on a printer in a pre-defined way or optionally control how it is printed; 

"Fax" which lets the user to send a fax of a user-specified document to a user-specified 
fax number and optionally control how it is faxed; 

"Play Video" which lets the user to play a user-specified video content and optionally 
control how it is played; 

"Play Audio" which lets the user to play the user-specified audio content and optionally 
control how it is played; 

"Digital Photo Frame" which lets the user to show a user-specified image content and 
optionally control how it is shown; 

"Location Determination" which automatically determines the location of a user-operated 

device; 

"Directory Publisher" which lets the user to choose one instance out of set of instances 
of the same type; 

'Telephone Dialer" which lets the user to dial a user-specified phone number; 

"TC Storage" which lets the user to store or retrieve files; 

"Copy to Removable Storage" which lets the user to store one or a set of files into 
predefined removable storage media and optionally control how they are saved; 

"Emailer" which lets the user to send email to a user-specified email address optionally 
with attached user-specified documents and optionally control how it is sent; 

"List chooser" which lets the user to choose an semantic instance from the list of 
semantic instances of the same type; 

"Property chooser" which lets the user to choose one of the properties from a semantic 
instance; 
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"Your input" which lets the user to input the values for the attributes of the semantic 

object; 

"Instance viewer" which lets the user to check and modify a semantic instance; 

"Local Instance Selector" which lets the user to select one or more semantic instances 
local to the computing device; 

"Semantic Instance Copier" which lets the user to copy the semantic instance into the 
local computing device; 

"L-Note" which lets the user to read and leave notes pertinent to a location; 

"Database publisher" which publishes a whole of or a part of a table from a database; 

"Place information" which provides information related to the place; and 

"Hosted Services" which uses the client-side browser functionalities to accomplish their 
tasks, including "Upload File" to let the user to upload a file from the client device, "URL to input" 
to let the user to input an URL, and "View on Browser" to let the user view the file on the client 
device. 

52. The computer-based system of claim 12, wherein the means for creating, 
removing, and managing services includes any one or combination of the following: 

means for making SDSs available through one or more of the discovery mechanisms, 
means for changing the discovery mechanism to use, 
means for holding the SDS, 

means for recovering the SDS to one of the discovery mechanisms, 
means for removing the SDSs from discovery mechanisms, and 
means for changing its publishing parameters including expiration, invocation limit, and 
access control. 

53. The computer-based system of claim 52, wherein the means for creating, 
removing, and managing services comprise any one or a combination of: 

"PIPE" which provides local or remote APIs to publish and optionally manage services, 
and optionally provides the user with user interfaces to publish and manage the 
services; 

"Semantic Instance Scrapers for Applications" which publishes the instances as they 
become available for the application and optionally lets the user manage the service provisions; 
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"Instance Selector with Instance Providing Service Creation" which lets the user publish 

the instance the user selected for the instance selector service and optionally lets the user 

manage the service provisions; 

'White hole" which lets the user publish objects from a local computing device as 

semantic instances through its user interface and optionally lets the user manage the service 

provisions; 

"Public Directory" which lets the user publish instances on a device and optionally lets 
the user manage the service provisions; 

"Bank" which lets the user publish instances through its service invocation and optionally 
lets the user manage the service provisions; 

"Image/Audio/Video Service Control" which lets the user publish instances created by or 
from the devices and optionally lets the user manage the service provisions; 

"Removable Media Publisher" which lets the user publish instances on the removable 
media inserted into a computing device and optionally lets the user manage the service 
provisions; and 

"Fax" which publishes the documents received and optionally lets the user manage the 
service provisions. 

54. A computer-based method comprising: 

performing task computing, thus enabling users to define tasks by combining available 
functionality and to execute such tasks. 

55. The computer-based method of claim 54, wherein available functionality 
originates in devices, computing applications and electronic services available through remote 
procedure calls including Web Services, UPnP, CORBA, RMI, RPC, DCE, DCOM or comprises 
previously defined tasks. 

56. The computer-based method of claim 55, wherein all available functionality is 
abstracted to the user as a service and each service is expressed in a service description 
language. 

57. The computer-based method of claim 56 wherein: 

each service has at least one semantic description associated with it, 
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each semantic description can be provided by any combination of the creator of the 

service, the owner of the service or some other third party, and 

the service can be made available or unavailable by making available or unavailable one 

or more of its semantic descriptions. 

58. The computer-based method of claim 57 further comprising: 
discovering available services; 

filtering the services based upon relevance of the discovered services to the user and 
her context; 

specifying the task possibly in collaboration with the user; 
executing the specified task; and 

optionally creating, removing, and managing services. 

59. The computer-based method of claim 58 wherein the specified tasks are saved, 
automatically, or as instructed by the user, as new available services. 

60. The computer-based method of claim 59, wherein the previously specified tasks 
saved as services are available to the user who created them, to all other users, or to any user- 
defined, or pre-defined group of users. 

61 . The computer-based method of claim 60, wherein the means for discovering 
services utilize a service discovery protocol, including any one or any combination of the 
following: 

UPnP, UDDI, Local Service Repository, Jini, Bluetooth SDP, Rendezvous, and InfraRed 

(IR). 

62. The computer-based method of claim 61 , wherein the filtering discovered 
services include any one or any combination of the following: 

User profile, Task at hand, User device characteristics, User location, User motion status, 
User network connectivity, User specified keywords, features of a set of services when 
considered as a whole, individual service features. 

63. The computer-based method of claim 62, wherein the specifying a task include 
any one or any combination of the following: 
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a planning-based, automated system, or 

an interactive user interface that supports any one or any combination of the following 
elements: visual, voice, text, Braille, tactile 

64. The computer-based method of claim 63, wherein the means for executing a 
specified task utilizes a service execution mechanism including any one or any combination of 
the following: Web Services calls, UPnP actions, CORBA, RMI, RPC, DCE, DCOM, InfraRed 
(IR), Local function call, and Local object call. 

65. The computer-based method of claim 64, wherein the creating, removing, and 
managing services include any one or any combination of the following: Graphical user 
interface, service execution, device interaction, events from the operating system of the user- 
operated device and applications executing on the user-operated device. 

66. A computer-readable medium storing a program which when executed by a 
computer causes the computer to execute the functions comprising: 

performing task computing, thus enabling users to define tasks by combining available 
functionality and to execute such tasks. 

67. The computer-readable medium of claim 66, wherein available functionality 
originates in devices, computing applications and electronic services available through remote 
procedure calls including Web Services, UPnP, CORBA, RMI, RPC, DCE, DCOM or comprises 
previously defined tasks. 

68. The computer-readable medium of claim 67, wherein all available functionality is 
abstracted to the user as a service and each service is expressed in a service description 
language. 

69. The computer-readable medium of claim 68, wherein: 

each service has at least one semantic description associated with it, 

each semantic description can be provided by any combination of the creator of the 

service, the owner of the service or some other third party, and 

the service can be made available or unavailable by making available or unavailable one 

or more of its semantic descriptions. 
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70. The computer-readable medium of claim 69, further comprising: 
discovering available services; 

filtering the services based upon relevance of the discovered services to the user and 
her context; 

specifying the task possibly in collaboration with the user; 
executing the specified task; and 

optionally creating, removing, and managing services. 

71 . The computer-readable medium of claim 70, wherein the specified tasks are 
saved, automatically, or as instructed by the user, as new available services. 

72. The computer-readable medium of claim 71 , wherein the previously specified 
tasks saved as services are available to the user who created them, to all other users, or to any 
user-defined, or pre-defined group of users. 

73. The computer-readable medium of claim 72, wherein the discovering services 
utilize a service discovery protocol, including any one or any combination of the following: 

UPnP, UDDI, Local Service Repository, Jini, Bluetooth SDP, Rendezvous, and InfraRed 

(IR). 

74. The computer-readable medium of claim 73, wherein the filtering discovered 
services include any one or any combination of the following: 

User profile, Task at hand, User device characteristics, User location, User motion status, 
User network connectivity, User specified keywords, features of a set of services when 
considered as a whole, individual service features. 

75. The computer-readable medium of claim 74, wherein the specifying a task 
include any one or any combination of the following: 

a planning-based, automated system, or 

an interactive user interface that supports any one or any combination of the following 
elements: visual, voice, text, Braille, tactile 
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76. The computer-readable medium of claim 75, wherein the executing a specified 
task utilizes a service execution mechanism including any one or any combination of the 
following: Web Services calls, UPnP actions, CORBA, RMI, RPC, DCE, DCOM, InfraRed (IR), 
Local function call, and Local object call. 

77. The computer-readable medium of claim 76, wherein the creating, removing, 
and managing services include any one or any combination of the following: Graphical user 
interface, service execution, device interaction, events from the operating system of the user- 
operated device and applications executing on the user-operated device. 
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